Skip to main content
SpringerLink
Log in

An adaptive competitive penalty method for nonsmooth constrained optimization

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

We present a competitive algorithm to minimize a locally Lipschitz function constrained with locally Lipschitz constraints. The approach is to use an ℓ 1 nonsmooth penalty function. The method generates second order descent directions to minimize the ℓ 1 penalty function. We introduce a new criterion to decide upon acceptability of a Goldstein subdifferential approximation. We show that the new criterion leads to an improvement of the Goldstein subdifferential approximation, as introduced by Mahdavi-Amiri and Yousefpour. Also, making use of our proposed line search strategy, the method always moves on differentiable points. Furthermore, the method has an adaptive behaviour in the sense that, when the iterates move on adequately smooth regions, the search directions switch exactly to the Shanno’s conjugate gradient directions and no subdifferential approximation is computed. The global convergence of the algorithm is established. Finally, an extensive comparison of the results obtained by our proposed algorithm with the ones obtained by some well-known methods shows significant reductions in number of function and gradient evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Clarke, F.H.: Optimization and nonsmooth analysis. Canadian Mathematical Society, Series of monographs and advanced texts. Wiley, New York (1983)

    Google Scholar 

  2. Shor, N.Z.: Minimization methods for non-differentiable functions. Springer Verlag, Berlin (1985)

    Book  MATH  Google Scholar 

  3. Kiwiel, K.C.: Methods of descent for nondifferentiable optimization. Lecture Notes in Mathematics 1133. Springer-Verlag (1985)

  4. Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth optimization: analysis and algorithms with applications to optimal control. World Scientific, Singapore (1992)

    Book  MATH  Google Scholar 

  5. Frangioni, A.: Generalized bundle methods. SIAM J. Optim. 14, 743–756 (2003)

    MathSciNet  Google Scholar 

  6. Gaudioso, M., Monaco, M.F.: A bundle type approach to the unconstrained minimization of convex nonsmooth functions. Math. Program. 23, 216–226 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  7. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II, Advanced Theory and Bundle Methods. Springer-Verlag, Berlin (1993)

    MATH  Google Scholar 

  8. Mifflin, R.: An algorithm for constrained optimization with semismooth functions. Math. Oper. Res. 2, 191–207 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  9. Wolfe, P.H.: A method of conjugate subgradients of minimizing nondifferentiable convex functions. Math. Program. Study 3, 145–173 (1975)

    Article  MATH  Google Scholar 

  10. Mahdavi-Amiri, N., Yousefpour, R.: An effective nonsmooth optimization algorithm for locally Lipschitz functions. J. Optim. Theory and Appl. 155, 180–195 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. Bagirov, A.M.: Continuous subdifferential approximations and their applications. J. Math. Sci. 115, 2567–2609 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  12. Bagirov, A.M., Ganjehlou, A.N.: A quasisecant method for minimizing nonsmooth functions. Optim. Methods Softw. 25, 3–18 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. Fuduli, A., Gaudioso, M., Giallombardo, G.: A DC piecewise affine model and a bundling technique in nonconvex nonsmooth minimization. Optim. Methods Softw. 19, 89–102 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  14. Fuduli, A., Gaudioso, M., Nurminski, E.A.: A splitting bundle approach for non-smooth non-convex minimization. Optimization 64, 1131–1151 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  15. Haarala, M., Miettinen, K., Mäkelä, M.M.: New limited memory bundle method for large-scale nonsmooth optimization. Optim. Methods Softw. 19, 673–692 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  16. Kiwiel, K.C.: Proximity control in bundle methods for convex nondifferentiable minimization. Math. Program. 46, 105–122 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  17. Lukšan, L., Vlček, J.: A bundle-Newton method for nonsmooth unconstrained minimization. Math. Program. 83, 373–391 (1998)

    MathSciNet  MATH  Google Scholar 

  18. Chen, X., Fukushima, M.: Proximal quasi-Newton methods for nondifferentiable convex optimization. Math. Program. 85, 313–334 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  19. Curtis, F.E., Overton, M.L.: A sequential quadratic programming algorithm for nonconvex nonsmooth constrained optimization. SIAM J. Optim. 22, 474–500 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  20. Karmitsa, N., Mäkelä, M.M., Ali, M.M.: Limited memory interior point bundle method for large inequality constrained nonsmooth minimization. Appl. Math. Comput. 198, 382–400 (2008)

    MathSciNet  MATH  Google Scholar 

  21. Goldstein, A.A.: Optimization of Lipschitz continuous functions. Math. Program. 13, 14–22 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  22. Nocedal, J., Wright, S.J.: Numerical optimization, 2nd ed. Springer Ser. Oper. Res., New York (2006)

    MATH  Google Scholar 

  23. Kiwiel, K.C.: An exact penalty function algorithm for non-smooth convex constrained minimization problems. IMA J. Numer. Anal. 5, 111–119 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  24. Polak, E., Mayne, D.Q., Wardi, Y.: On the extension of constrained optimization algorithms from differentiable to nondifferentiable problems. SIAM J. Control Optim 21, 179–203 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  25. Kiwiel, K.C.: An aggregate subgradient method for nonsmooth convex minimization. Math. Program. 27, 320–341 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  26. Kiwiel, K.C.: Exact penalty functions in proximal bundle methods for constrained convex nondifferentiable minimization. Math. Program. 52, 285–302 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  27. Herskovitz, J.: Feasible direction interior point technique for nonlinear optimization. J. Optim. Theory and Appl. 99, 121–146 (1998)

    Article  MathSciNet  Google Scholar 

  28. Herskovitz, J., Santos, G.: On the computer implementation of feasible direction interior point algorithms for nonlinear optimization. Struct. Optim. 14, 165–172 (1997)

    Article  Google Scholar 

  29. Haarala, M.: Large-scale nonsmooth optimization: variable metric bundle method with limited memory. Ph.D. Thesis, University of Jyväskylä. Department of Mathematical Information Technology (2004)

  30. Burke, J.V., Lewis, A.S., Overton, M.L.: A robust gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 15, 751–779 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  31. Shanno, D.F.: Conjugate gradient methods with inexact searches. Math. Oper. Res. 3, 244–256 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  32. Curtis, F.E., Que, X.: An adaptive gradient sampling algorithm for nonsmooth optimization. Optim. Methods and Softw. 28, 1302–1324 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  33. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II, A series of comprehensive studies in mathematics 299, Springer-Verlag, Berlin, Heidelberg, New York (1993)

  34. Rosenberg, E.: Exact penalty functions and stability in locally Lipschitz programming. Math. Program. 30, 340–356 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  35. Beale, E.M.L.: A Derivation of Conjugate Gradients. In: Lootsma, F.A (ed.) Numerical Methods for Nonlinear Optimization, pp. 39–43. Academic Press, London (1972)

  36. Lewis, A.S., Overton, M.L.: Nonsmooth optimization via quasi-Newton methods. Math. Program. 141, 135–163 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  37. Kiwiel, K.C.: Convergence of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 18, 379–388 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  38. Davidon, W.C.: Optimally conditioned optimization algorithms without line searches. Math. Program. 9, 1–30 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  39. Shanno, D.F.: On the convergence of a new conjugate gradient algorithm. SIAM J. Numer. Anal. 15, 1247–1257 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  40. Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  41. Bertsimas, D., Brown, D.B., Caramanis, C.: Theory and applications of robust optimization. SIAM Rev. 53, 464–501 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Mathematical Sciences, Sharif University of Technology, Tehran, Iran

    N. Mahdavi-Amiri & M. Shaeiri

Authors
  1. N. Mahdavi-Amiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Shaeiri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to N. Mahdavi-Amiri.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mahdavi-Amiri, N., Shaeiri, M. An adaptive competitive penalty method for nonsmooth constrained optimization. Numer Algor 75, 305–336 (2017). https://doi.org/10.1007/s11075-016-0208-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-016-0208-6

Keywords

Search

Navigation